By the end of this tutorial, you will understand:
Essential for
Critical thinking
Research literacy
Evaluating claims
Making informed decisions
Everyone who wants to think more clearly and critically:
No prior knowledge required - we start from first principles!
We are not naturally good at reasoning
- 🧠 We see patterns that aren’t there
- 📊 We’re terrible with probability
- 💭 We prefer confirming our beliefs to testing them
- 😨 We rely on emotion over evidence
- 🎲 We misunderstand randomness
Science is the solution
- A systematic method to overcome our biases
- A process for reliable knowledge
- A safeguard against self-deception
Before diving into details, let’s start with a working definition:
Science is a methodological process used to acquire knowledge about the world based on empirical evidence.
Key components
- Methodological: Systematic, not haphazard
- Process: Ongoing, not finished
- Empirical evidence: Observations, not speculation
- About the world: Reality, not abstract logic
Question Why can’t we just figure things out by thinking?
Answer For some things, we can!
Formal sciences (logic, mathematics) work this way:
- Start with axioms (basic assumptions)
- Apply logical operations
- Derive necessary conclusions
Example - Valid Logical Inference
Premise 1: Socrates is a human being
Premise 2: All humans are mortal
Conclusion: Therefore, Socrates is mortal This works because
- If premises are true
- And logic is valid
- Then conclusion must be true
Problem Logic can’t tell us which possible world is our world.
Example
Possible world 1: I raise my left arm after counting to 3
Possible world 2: I raise my right arm after counting to 3
Possible world 3: I raise neither arm after counting to 3 All three are logically coherent!
Question Which one actually happened?
Answer We need empirical evidence to know!
(As a side note: I only counted to two and raised neither arm.)
Question If we need evidence, why not just observe carefully?
Answer Because humans are systematically biased observers!
The rest of this tutorial demonstrates this problem.
What we fear ≠ What actually harms us
Example 1: Stranger Danger
- Fear: Unknown people will harm us or our children
- Reality: Most violence occurs within families/known contacts
- Result: Misdirected protective efforts
Example 2: Sharks vs. Cows
- Fear: Shark attacks (dramatic, memorable)
- Reality: Cows kill ~20 people/year in US; sharks kill <1
- Mosquitoes: Kill ~700,000 people/year globally
- Result: Movie “Jaws” terrifies; cows seem harmless
Why?
- Emotional narratives override statistics
- Vivid stories more memorable than data
- Evolution favored quick emotional responses
Definition Seeking evidence that confirms what we already believe, ignoring contradictory evidence.
Why it’s a problem
- Prevents learning
- Reinforces errors
- Creates echo chambers
- Hinders scientific progress
Example We’ll see this demonstrated in the Wason Selection Task later.
Most people are shocked by how bad we are at probability and statistics. Let’s demonstrate with famous examples.
The Setup
Monty Hall hosted the TV show “Let’s Make a Deal.” The game:
Should you switch doors?
Think about it before reading on!
Most people think “It doesn’t matter—50/50 chance either way.”
This is WRONG!
You should ALWAYS switch!
Switching gives you a 2/3 chance of winning.
Staying gives you only a 1/3 chance.
Initial choice
- Door 1: 1/3 chance of prize
- Doors 2 & 3 together: 2/3 chance of prize
After Monty opens Door 3 (showing goat)
- Door 1: Still 1/3 chance (your initial choice hasn’t changed)
- Door 2: Now has the full 2/3 chance!
Why? Monty’s action concentrates the 2/3 probability onto the unopened door you didn’t choose.
| Step | Diagram | Explanation |
|---|---|---|
| Initial choice |  |
You pick Door 1 (1/3 chance). Other doors together have 2/3 chance. |
| Monty reveals |  |
Monty shows goat behind Door 3. The 2/3 probability concentrates on Door 2. |
Imagine 20 doors instead of 3
| Step | Explanation |
|---|---|
| Initial | You pick Door 1 (1/20 chance). Other 19 doors have 19/20 chance. |
| Monty reveals | Monty opens 18 doors (all goats), leaving one unopened door. |
| Question | Would you switch to that one remaining door? |
Of course you would! The 19/20 probability has concentrated on that one door.
Same principle with 3 doors—just less obvious.
Simulation Monty Hall Interactive
Run it 100 times:
- Staying strategy: ~33% win rate
- Switching strategy: ~67% win rate
This demonstrates how bad our intuition is with probability!
How many people need to be in a room for there to be a 50% chance that two share a birthday?
Think about your answer before reading on!
Most people guess 100+ people, or even 183 (half of 365)
The correct answer Only 23 people!
In a room of 23 people, there’s a 50.7% chance two share a birthday.
Most people are shocked by this!
We’re good at
- Simple addition, subtraction, multiplication
- Linear thinking
We’re terrible at
- Exponential growth
- Combinatorial explosion
- Probability calculations
Direct approach (very hard)
- List all possible birthday combinations
- Count matching pairs
- Calculate probability
Smart approach (much easier)
- Calculate probability that everyone has different birthdays
- Subtract from 1
Question restated What’s the probability at least 2 people share a birthday?
Easier question What’s the probability that all 23 have different birthdays?
Person 1: 365/365 (any birthday)
Person 2: 364/365 (must differ from person 1)
Person 3: 363/365 (must differ from 1 and 2)
...
Person 23: 343/365 (must differ from all 22 others)
Probability all different = (365/365) × (364/365) × (363/365) × ... × (343/365)
= 0.4927
Probability at least one match = 1 - 0.4927 = 0.5073 Result 50.73% chance of at least one match!
# Number of people
n <- 23
# Days in year
days_in_year <- 365
# Probability all different birthdays
prob_all_different <- prod((days_in_year - 0:(n - 1)) / days_in_year)
# Probability at least one match
prob_match <- 1 - prob_all_different
prob_match
# [1] 0.5072972 Question What’s the probability of a birthday match with 73 people?
n <- 73
days_in_year <- 365
prob_all_different <- prod((days_in_year - 0:(n - 1)) / days_in_year)
prob_match <- 1 - prob_all_different
prob_match
# [1] 0.9999919 With 73 people, it’s 99.999% certain two share a birthday!
We systematically underestimate how quickly probabilities accumulate.
Why this matters for science
- Can’t rely on intuition for statistical reasoning
- Need systematic methods
- Math and probability are counterintuitive
- Must check calculations, not trust “gut feeling”
A ball and bat together cost $1.10.
The bat costs $1.00 more than the ball.
How much does the ball cost?
Most people immediately think 10 cents
This is WRONG!
Check If ball = 10¢ and bat = $1 more than ball:
- Ball = 10¢
- Bat = $1.10
- Total = $1.20 ✗ (not $1.10!)
Ball costs 5 cents.
Check
- Ball = 5¢
- Bat = $1.05 (which is $1 more than ball ✓)
- Total = $1.10 ✓
Daniel Kahneman’s framework (Kahneman 2011)
System 1 (Fast Thinking)
- Automatic, effortless
- Intuitive, quick
- Pattern-based
- Often wrong on complex problems
System 2 (Slow Thinking)
- Deliberate, effortful
- Analytical, careful
- Logic-based
- More accurate but requires effort
Problem System 1 answers first, System 2 is lazy!
Science forces us to use System 2 thinking
- Deliberate methodology
- Careful observation
- Systematic analysis
- Checked reasoning
Science is expensive (time/effort) but accurate!
The Experiment (B.F. Skinner, 1948)
Setup
- Pigeons in boxes with food dispensers
- Food delivered at random intervals
- No connection to pigeon behavior
Result
- Pigeons developed “superstitious” behaviors
- One pigeon turned in circles
- Another pecked at corners
- Each pigeon repeated whatever it was doing when food appeared
Interpretation
- Pigeons assumed causation from correlation
- Behavior became “reinforced” despite being unrelated to food
- Random reward schedule created superstition
Part 1 Skinner’s Pigeons (1/2)
Part 2 Skinner’s Pigeons (2/2)
Athletes
- Not shaving during playoffs
- Wearing “lucky” socks
- Pre-game rituals
Students
- Lucky pens for exams
- Specific study spots
- Pre-test rituals
Gamblers
- “Hot” slot machines
- Lucky numbers
- Betting systems
All share the same mechanism
- Random co-occurrence of behavior and outcome
- Assumed causation
- Behavior reinforced by occasional success
Pareidolia Seeing meaningful patterns (especially faces) in random stimuli
1. Face on Mars (1976)
- Viking 1 orbiter photo showed “face”
- Actually just lighting and shadows on rock formation
- Higher resolution photos (2001) showed ordinary mesa
2. Jesus on Toast
- Religious figures in food patterns
- Burn marks, mold, textures
- Multi-million dollar eBay sales!
3. Man in the Moon
- Dark lunar maria form face-like pattern
- Different cultures see different patterns
- Rabbit (East Asian), woman (Māori), etc.
Evolutionary explanation (Bruce Hood, Cardiff University)
Benefits of face-detection
- Quickly identify friends vs. foes
- Read emotions and intentions
- Detect predators or threats
- Social bonding and communication
Cost of false alarms
- Low cost: Thought rock was face → no harm
- High cost: Thought face was rock → eaten by predator!
Result Evolution favored over-sensitive face detection
Scenario
A professor offers you $10 to wear a sweater for one minute.
Question 1 Would you do it?
Most people Yes!
Plot twist The professor mentions the sweater belonged to a notorious serial killer.
Question 2 Would you still wear it?
Typical responses
- Some people refuse
- Others feel uncomfortable but might still do it
- Many report it “feels different” now
Why?
Irrational belief The sweater has been “contaminated” or acquired some essence
Rational fact It’s just cloth—molecules of cotton/wool
Evolutionary explanation
- Ancestors who avoided items from diseased/dead people
- …were less likely to contract diseases
- …survived and reproduced more
- Result We inherited cautious behavior toward “contaminated” items
Even though irrational, this behavior had evolutionary advantages!
Anthropocentric bias Assuming the world appears to all creatures as it appears to us
Alternative name Experiential Realism
Human vision
- Visible spectrum: ~400-700 nm wavelength
- Colors: Red, orange, yellow, green, blue, violet
Bee vision
- Visible spectrum: ~300-650 nm
- See ultraviolet (UV) light
- Don’t see red (appears black)
What this means
For bees
- Flowers stand out against background (like stars in night sky)
- UV patterns guide to nectar (invisible to us)
- World looks dramatically different
Many flowers evolved UV patterns specifically for bee vision!
“However, the parts of this external reality to which we have access are largely constrained by the ecological niche we have adapted to and the nature of our embodiment. In other words, language does not directly reflect the world. Rather, it reflects our unique human construal of the world: our ‘world view’ as it appears to us through the lens of our embodiment.”
— Evans and Green (2006, 46)
Key insight Our perception is species-specific, not objective reality!
Instructions
What happens?
Result
- Left dune appears greenish
- Right dune appears reddish
Why?
- Red-sensitive neurons fatigued (depleted neurotransmitters)
- Green-sensitive neurons fatigued
- When viewing neutral sand:
- Left side: red neurons can’t fire → see green
- Right side: green neurons can’t fire → see red
Lesson What we “see” depends on our neurophysiology, not just external reality!
Gestalt German word meaning “form” or “shape”
Gestalt psychology Studies how we perceive wholes from parts
Question Do you see a triangle?
Answer There is no triangle!
What’s happening
- Brain fills in missing information
- Creates coherent “Gestalt” from incomplete data
- We cannot help but see the triangle
Re-arranged
Now you see Pac-Man!
Same elements, different arrangement → different perception
Key principles
All demonstrate Perception is active construction, not passive recording
Instructions Look at these upside-down faces. Does one look odd?
Probably the right image seems slightly strange.
Now look at them right-side up
Most people are shocked!
Why?
When upside-down
- Brain doesn’t process face normally
- Local distortions less noticeable
- “Good enough” processing
When right-side-up
- Full face-processing activated
- Grotesque distortions obvious
- Brain normalizes faces based on experience
Lesson Perception adjusts based on context and expectation!
Question Do you see a face?
Answer There is no face—just random elements
But we see one anyway!
Why?
- Face recognition crucial for survival
- Better to see face when there isn’t one
- …than miss face when there is one!
- Evolution favored over-sensitive face detection
What symbol is in the red circle?
Most people see Letter “B”
Same symbol in different context
Now most people see Number “13”
Key insight
Same visual stimulus → Different interpretation based on context
Context determines categorization!
Logical fallacy Flawed, deceptive, or false argument that can be proven wrong with reasoning
Why they matter
- Very common in everyday discourse
- Everyone is susceptible
- Prevent accurate conclusions
- Undermine rational debate
- Must be recognized and avoided
What it is
- Seeking evidence that confirms existing beliefs
- Ignoring contradictory evidence
- Selectively reporting favorable results
Example
Person: "Vaccines cause autism!"
Evidence: 1 study (retracted, fraudulent) says yes
100+ studies say no
Action: Cites the 1 study, ignores the 100+ Why it’s bad
- Prevents learning
- Reinforces errors
- Hinders scientific progress
Scientific solution
- Actively seek disconfirming evidence
- Report all results (positive and negative)
- Pre-register analysis plans
What it is
- Attacking the person instead of their argument
- Using personal insults instead of logic
- Mudslinging
Examples
❌ "You can't trust their climate research—they're a leftist!"
❌ "His statistics are wrong because he's funded by Big Pharma!"
❌ "She's just saying that because she's young and naive." What’s wrong
- Person’s character ≠ argument’s validity
- Deflects from actual issues
- Prevents rational discussion
Correct approach
"Their climate model has these specific flaws: [list flaws]"
"The statistics use an inappropriate test because [explain]"
"Her argument overlooks this evidence: [cite evidence]" What it is
- Misrepresenting opponent’s position
- Making it easier to attack
- Defeating weak version instead of actual argument
Example
Person A: "We should have some gun regulations to reduce violence."
Person B: "You want to ban all guns and leave people defenseless
against criminals! That's absurd!"
[Person A never said "ban all guns"—that's a straw man] Why it’s called “straw man”
- Straw man = easy to knock down
- Like fighting a scarecrow instead of real opponent
- Creates appearance of winning without addressing real argument
How to avoid
- Accurately represent opponent’s actual position
- Ask for clarification if uncertain
- Address strongest version of argument
What it is
- Claiming something is true because it hasn’t been proven false
- Or vice versa
Examples
❌ "No one has proven aliens don't exist, so they must be real!"
❌ "Science can't explain consciousness, therefore it must be supernatural!"
❌ "We don't know how the universe began, so God must have created it!" Why it’s wrong
- Lack of evidence ≠ evidence of absence
- Burden of proof on person making claim
- Many things unknown—doesn’t make all explanations equally valid
Correct reasoning
"We don't have evidence for aliens, so we should remain agnostic."
"We don't fully understand consciousness, so we need more research."
"Multiple hypotheses about universe's origin remain scientifically viable." What it is
- Presenting only two options
- When actually more exist
- Usually presenting extremes
Examples
❌ "America: Love it or leave it!"
❌ "You're either with us or against us!"
❌ "Either we cut all social programs or the economy collapses!" Why it’s manipulative
- Polarizes discussion
- Eliminates middle ground
- Forces choice between extremes
- Obscures nuanced solutions
Reality
Middle positions exist
Partial solutions possible
Compromise feasible
Multiple options available What it is
- Claiming one action inevitably leads to extreme consequences
- Without evidence for the causal chain
- Each step supposedly leads inexorably to next
Examples
❌ "If we allow gay marriage, next people will marry animals!"
❌ "If we ban one gun, soon they'll ban all guns!"
❌ "If I don't go to this party, I'll have no friends, fail school,
and end up homeless!" When it’s legitimate
- If evidence supports each step in chain
- If mechanism for progression is demonstrated
- If historical precedent exists
When it’s a fallacy
- No evidence for progression
- Jumps to extreme without justification
- Uses fear instead of logic
What it is
- Conclusion is hidden in the premise
- Assumes what you’re trying to prove
- Argument repeats itself
Examples
❌ "The Bible is true because it says so in the Bible."
❌ "I'm trustworthy because I say I'm trustworthy."
❌ "This law is good because it promotes good values."
[What makes the values good?] Why it fails
- Doesn’t actually prove anything
- Just restates assumption
- No new information added
Valid argument structure
Independent premises → Conclusion
Evidence → Inference
Data → Interpretation What it is
- Introducing irrelevant information
- To distract from real issue
- Shifting focus to easier/safer topic
Example
Journalist: "Why did the government waste millions on this failed project?"
Politician: "Let me tell you about all the great schools we've built!
Education is so important, don't you agree?"
[Didn't answer question about waste—introduced different topic] Why it works
- People follow new conversational direction
- Original question forgotten
- Difficult topic avoided
How to counter
- Recognize the shift
- Return to original question
- Don’t follow the distraction
What it is
- Continuing because of already invested resources
- Even when additional costs outweigh benefits
- “Can’t waste what I’ve already put in!”
Examples
❌ "I've watched 5 seasons of this show—I have to finish it!"
[Even though you're not enjoying it]
❌ "I've already spent $500 fixing this car—I should keep fixing it!"
[Even though it needs $2000 more and is worth $1000]
❌ "We've invested 5 years in this relationship—can't give up now!"
[Even though both people are miserable] Why it’s irrational
- Past costs are gone (sunk)
- Should only consider future costs vs. benefits
- Past investment is irrelevant to future decisions
Rational approach
Evaluate: Future costs vs. Future benefits
Ignore: Past costs (already gone)
Ask: "If I were starting fresh, would I begin this?" Without awareness of fallacies
- ❌ Reach wrong conclusions
- ❌ Defend indefensible positions
- ❌ Waste resources
- ❌ Spread misinformation
With systematic thinking
- Recognize flawed reasoning
- Avoid cognitive biases
- Reach accurate conclusions
- Build reliable knowledge
Science is the antidote to fallacious thinking!
The Setup
You see four cards with letters on one side, numbers on the other:
| Card 1 | Card 2 | Card 3 | Card 4 |
| A | K | 2 | 7 | The Rule “If vowel on one side, then even number on other side”
Which card(s) must you turn over to test whether the rule is true?
Think carefully before reading on!
Most common answer Cards 1 (A) and 3 (2)
This is WRONG!
Correct answer Cards 1 (A) and 4 (7)
Why?
Card 1 (A)
- ✅ Must check: Vowel, so must have even number
- If odd number on back → rule is false
Card 2 (K)
- ✗ No need to check: Consonant
- Rule says nothing about consonants
- Could have even or odd—both OK
Card 3 (2)
- ✗ No need to check: Even number
- Rule says nothing about what’s on other side of even numbers
- Could have vowel or consonant—both OK
Card 4 (7)
- ✅ Must check: Odd number
- If vowel on back → rule is false
- (Rule says vowels must have even numbers)
What this demonstrates
Confirmation bias!
People test cases that could confirm the rule (vowel, even number)
…instead of cases that could falsify it (vowel?, odd number)
Scientific thinking requires
- Testing what could prove you wrong
- Not just seeking confirmation
- Actively trying to falsify
I have a rule for generating numbers. Here are three numbers following my rule
| Number 1 | Number 2 | Number 3 | Number 4 |
| 1 | 2 | 4 | ? | You can propose one number, and I’ll tell you if it follows my rule.
What number would you propose?
What rule do you think I have in mind?
Typical responses
First guess 8
Proposed rule “Double the previous number”
My response Yes, 8 follows my rule!
But This is NOT my rule!
Second guess 16
Proposed rule “Square the previous number”
My response Yes, 16 follows my rule!
But This is STILL NOT my rule!
Better approach
Test numbers that would disconfirm your hypothesis
- Try 3 (doesn’t fit doubling rule)
- Try 7 (doesn’t fit doubling rule)
- Try 10 (doesn’t fit doubling rule)
My actual rule “Each number must be larger than the previous”
What this demonstrates
Confirmation bias again!
People propose numbers that confirm their hypothesis
…instead of numbers that would falsify it
Science requires
- Testing what would prove you wrong
- Challenging your own ideas
- Seeking disconfirming evidence
We are systematically biased
- See patterns in randomness
- Terrible with probability
- Prefer emotion to evidence
- Seek confirmation, not falsification
- Susceptible to logical fallacies
- Perceive world through human lens
These biases
- Often have evolutionary basis
- Are part of human nature
- Constantly operate
- Lead us astray systematically
Science is the solution
- Systematic methodology
- Controlled observation
- Statistical analysis
- Peer review
- Replication
- Falsification
Science protects us from ourselves!
Science is
An unbiased, fundamentally methodological enterprise that aims at building and organizing knowledge about the empirical world in the form of falsifiable explanations and predictions by means of observation and experimentation.
Key components
Science is
The effort to understand how the universe works through the scientific method, with observable evidence as the basis of that understanding.
Definition Examine phenomena of reality through scientific method
Goal Explain and/or predict empirical phenomena
Examples
- Biology (life processes)
- Physics (matter and energy)
- Chemistry (substances and reactions)
- Psychology (behavior and cognition)
- Sociology (social phenomena)
- Linguistics (language phenomena)
- Astronomy (celestial objects)
Method
- Observe reality
- Form hypotheses
- Test predictions
- Refine theories
Definition Examine abstract systems using axiomatic reasoning
Goal Logical coherence and internal consistency
Examples
- Mathematics
- Formal logic
- Theoretical computer science
- System theory
- Chaos theory
- Formal linguistics
Method
- Start with axioms
- Apply logical operations
- Derive theorems
- Prove consistency
Formal sciences
- Can prove statements true
- Based on logic and axioms
- Don’t require empirical evidence
- Abstract systems
Empirical sciences
- Cannot prove—only falsify
- Based on observation
- Require empirical evidence
- Real-world phenomena
Basic cycle
1. Observe phenomenon
↓
2. Ask question
↓
3. Form hypothesis
↓
4. Design test
↓
5. Collect data
↓
6. Analyze results
↓
7. Draw conclusions
↓
8. Refine hypothesis → Back to step 3 1. Make an Observation
- Notice something interesting
- Identify a pattern or problem
- Example: “My keys are missing!”
2. Formulate Research Question
- What do you want to know?
- Make it specific
- Example: “Where are my keys?”
3. Review Literature
- What do we already know?
- Has this been studied before?
- Example: “Where have I lost keys before?”
4. Form Hypothesis (H₁)
- Testable prediction
- Based on observation and literature
- Example: “My keys are on the table by the TV”
5. Form Null Hypothesis (H₀)
- Opposite of your prediction
- What you’re trying to disprove
- Example: “My keys are NOT on the table by the TV”
6. Determine Significance Level
- How certain do you need to be?
- Typically α = 0.05 (5% chance of error)
- Balance Type I and Type II errors
7. Design Study
- How will you test hypothesis?
- What data will you collect?
- What methods will you use?
- Example: “I will go check the table by the TV”
8. Collect Data
- Execute your design
- Record observations
- Example: “I checked—keys not there”
9. Analyze Data
- Apply appropriate statistical tests
- Determine if results are significant
- Calculate effect sizes
10. Draw Conclusions
- Can you reject H₀?
- What does this mean for H₁?
- Example: “Keys must be somewhere else”
11. If H₀ Not Rejected
- Form new hypothesis
- Repeat cycle
- Example: “Maybe keys are in my coat pocket”
Complete scenario
Observation: My keys are gone!
Question: Where are my keys?
Literature: I've lost them on the TV table before
Hypothesis (H₁): Keys are on TV table
Null (H₀): Keys are NOT on TV table
Design: I will check the TV table
Data: I checked—no keys there
Analysis: H₀ cannot be rejected
Conclusion: Keys must be elsewhere
New H₁: Keys are in coat pocket
[Repeat cycle...] 
The phenomenon (1891-1904)
Clever Hans could apparently:
- Do arithmetic
- Answer questions in German
- Spell words
- Tell time
- Understand human language
Method
- Owner (Wilhelm von Osten) would ask questions
- Hans would tap his hoof the correct number of times
- Appeared to count, calculate, and comprehend
Public reaction
- Sensation across Europe
- Huge crowds
- Scientific investigations
What experts observed
- ✓ No apparent trickery
- ✓ Worked with different questioners
- ✓ Von Osten genuinely believed in Hans
- ✓ No obvious signals being given
Conclusion of many scientists
- Horse understands human language
- Horse can perform mathematics
But some remained skeptical…
Oskar Pfungst (1907) Psychologist who investigated systematically
Key experiments
Experiment 1: Questioner knows answer
- Result Hans answers correctly
Experiment 2: Questioner doesn’t know answer
- Result Hans cannot answer
Experiment 3: Hans can see questioner
- Result Hans answers correctly
Experiment 4: Hans cannot see questioner (blinders)
- Result Hans cannot answer
What Pfungst found
Hans was responding to involuntary micro-movements of questioners:
How it worked
1. Questioner asks question requiring numerical answer
2. Questioner unconsciously tenses up
3. Hans starts tapping
4. As Hans approaches correct number…
5. Questioner unconsciously relaxes slightly
6. Hans stops tapping
7. Appears to have “known” the answer!
The cues were
- Tiny muscle tensions
- Slight postural shifts
- Minor breathing changes
- Completely unconscious
What it teaches us
Applications to science
Double-blind experiments
- Neither subject nor experimenter knows condition
- Prevents unconscious cuing
- Standard in medical trials
Control conditions
- Test what happens without treatment
- Isolate causal factors
- Essential for valid conclusions
Systematic methodology
- Follow rigorous procedures
- Control for confounds
- Don’t rely on intuition
The “Clever Hans effect” now refers to:
- Unconscious experimenter influence
- Need for proper blinding
- Importance of controls
- Why methodology matters
Who was he?
- Austrian-British philosopher
- Major figure in philosophy of science
- Defender of liberal democracy
- Critic of totalitarianism
Key contribution Understanding how science actually works
Traditional view of science
Make many observations
↓
Find pattern
↓
Prove general law Popper’s insight This doesn’t actually work!
Why not?
Example
Observation 1: Swan 1 is white
Observation 2: Swan 2 is white
Observation 3: Swan 3 is white
...
Observation 10,000: Swan 10,000 is white
Conclusion: All swans are white Problem No number of white swans proves all swans are white!
But One black swan disproves it!
In empirical science
Cannot prove theories true
- Infinite number of possible observations
- Always possibility of contradictory evidence
- Observations limited, reality unlimited
Can prove theories false
- Single contrary observation
- Demonstrates theory incorrect
- Immediate and definitive
Therefore
Science advances by falsification, not verification!
A theory is scientific if and only if it is falsifiable.
Falsifiable means:
- Can imagine observation that would prove it wrong
- Makes predictions that could be tested
- Takes empirical risk
Not falsifiable
- Cannot be tested
- Compatible with any observation
- Not scientific (even if true!)
Examples
Falsifiable (Scientific)
✓ "All swans are white"
[Falsified by: Black swan]
✓ "Earth orbits the Sun"
[Falsifiable by: Stellar parallax measurements]
✓ "Smoking causes cancer"
[Falsifiable by: Epidemiological studies] Not Falsifiable (Not Scientific)
✗ "God exists"
[What observation would disprove?]
✗ "Everything happens for a reason"
[Compatible with any outcome]
✗ "This dream symbolizes your unconscious desires"
[Cannot be tested] Popper’s famous critique
Psychoanalysis (Freud)
- Any behavior confirms theory
- Contradictory evidence “reinterpreted”
- Makes no falsifiable predictions
- Therefore: Not scientific
Example
Theory: You have unconscious Oedipal complex
Evidence 1: You're close to your mother
→ Confirms theory!
Evidence 2: You're distant from your mother
→ Confirms theory! (You're repressing feelings)
[Every outcome confirms theory = not falsifiable] Popper’s evolutionary analogy
Biological evolution
Genetic variation
↓
Natural selection
↓
Survival of fittest
↓
Adaptation Scientific progress
Theory variation (conjectures)
↓
Empirical testing
↓
Survival of best-tested theories
↓
Better understanding Key parallels
Biology
- Genes that work → survive
- Genes that fail → eliminated
- “Fitness” = survival value
- No guarantee of future success
Science
- Theories that work → retained
- Theories that fail → rejected
- “Fitness” = withstanding tests
- No proof of absolute truth
Both involve
- Variation and selection
- Error elimination
- Progressive adaptation
- No final perfection
When designing research
Good questions (falsifiable)
✓ “Does treatment X reduce symptom Y?”
✓ “Are these two variables correlated?”
✓ “Does this theory predict observation Z?”
Bad questions (not falsifiable)
✗ “What is the meaning of life?”
✗ “Is this art beautiful?”
✗ “What does this symbol represent?”
When analyzing
- Don’t just seek confirming evidence
- Actively try to falsify your hypothesis
- Strongest theories = those that survive many attempts to disprove
When evaluating claims
- Ask: “What would prove this wrong?”
- If nothing could → not scientific
- If clear test exists → scientific (even if untested)
Linguistics is the scientific study of language or individual languages.
Linguists aim to
- Uncover systems behind language
- Describe these systems
- Explain them theoretically
- Model them formally
Empirical linguistics
- Studies language through observation
- Collects data from real language use
- Tests hypotheses about language
- Describes rather than prescribes
Descriptive vs. Prescriptive
Descriptive (Scientific)
✓ “English speakers say ‘ain’t’ in casual conversation”
✓ “Children acquire negation in stages”
✓ “This dialect has these features”
Prescriptive (Not Scientific)
✗ “You shouldn’t say ‘ain’t’”
✗ “Never end a sentence with a preposition”
✗ “This is the ‘correct’ way to speak”
Example: Language acquisition
1. Observation:
Children seem to learn grammar without explicit instruction
2. Question:
How do children acquire language?
3. Literature:
Chomsky: Universal Grammar hypothesis
Tomasello: Usage-based approach
4. Hypothesis:
Children extract patterns from input through frequency tracking
5. Design:
Present children with artificial language
Track which patterns they learn
Manipulate input frequency
6. Data:
Record children's productions
Count errors and correct forms
7. Analysis:
Compare learning rates for high vs. low frequency patterns
8. Conclusion:
Higher frequency → faster learning
Supports usage-based hypothesis
9. Refine:
Test with different age groups
Try different complexity levels
[Repeat cycle] Given what you’ve learned, explain belief in ghosts.
What cognitive biases and perceptual phenomena might contribute?
Multiple factors contribute
1. Pareidolia
- Seeing faces/figures in shadows, mist, patterns
- Brain over-interprets ambiguous stimuli
- Evolved tendency to detect agents
2. Confirmation bias
- Selectively remember “evidence” for ghosts
- Forget explained phenomena
- Seek confirming experiences
3. Sleep paralysis
- REM intrusion into wakefulness
- Cannot move, vivid hallucinations
- Feels like presence in room
4. Infrasound
- Very low frequency sounds (<20 Hz)
- Can cause feeling of presence
- Nausea, anxiety, fear
- Common in old buildings
5. Pattern-seeking
- Assign agency to random events
- See intention in coincidence
- Assume effects have intentional causes
6. Cultural transmission
- Stories reinforce beliefs
- Social learning of “ghost signs”
- Shared interpretive frameworks
7. Emotional factors
- Grief seeking connection to deceased
- Fear in unfamiliar places
- Heightened awareness in darkness
All these factors can create genuine ghost experiences without actual ghosts!
Someone says “My grandfather smoked a pack of cigarettes and drank a bottle of whiskey every day and lived to 95. So smoking and drinking don’t harm your health.”
What’s wrong with this reasoning?
Multiple problems
1. Sample size of 1
- One person ≠ general pattern
- Could be exceptional case
- Unusual cases memorable, not representative
2. Selection bias
- Only telling story because unusual
- Wouldn’t mention if grandfather died young
- Survivors more visible than victims
3. Confounds
- Maybe grandfather had lucky genes
- Maybe other health behaviors compensated
- Maybe environment was protective
- Unknown factors
4. Not representative
- Need systematic, unbiased sampling
- Large sample size
- Control for confounds
- Statistical analysis
Correct approach
- Large-scale epidemiological studies
- Compare smokers vs. non-smokers
- Control for age, genetics, other factors
- Measure health outcomes objectively
Result of proper studies
- Smoking: 15x higher lung cancer risk
- Heavy drinking: Liver disease, cancer, heart disease
- Overwhelming evidence of harm
Single anecdote cannot overturn systematic evidence!
Apply the scientific circle to studying the Loch Ness Monster.
How would you investigate this claim scientifically?
Scientific investigation
1. Observation
- Reports of large creature in Loch Ness
- Photos (low quality, disputed)
- Sonar readings (ambiguous)
2. Question
- Does a large unknown creature inhabit Loch Ness?
3. Literature Review
- Previous expeditions (multiple, 1930s-present)
- Sonar surveys (inconclusive)
- Environmental studies (ecology of loch)
- Analysis of photos (many hoaxes)
4. Hypothesis (H₁)
- A large unknown aquatic animal lives in Loch Ness
5. Null Hypothesis (H₀)
- No such creature exists (sightings explained by known phenomena)
6. Predictions from H₁
- Creature should appear on comprehensive sonar survey
- Should find environmental traces (feces, shed skin, etc.)
- Corpse should eventually surface
- DNA in water samples
- Population needed for breeding (multiple animals)
7. Design Study
- Comprehensive sonar mapping
- eDNA sampling (environmental DNA)
- Underwater cameras
- Systematic surface monitoring
- Geological/ecological assessment
8. Conduct Study
- Operation Deepscan (1987): Full loch sonar survey
- DNA studies (2019): Analyzed water samples
- Multiple expeditions with modern technology
9. Results
- No large creature detected on sonar
- No unknown DNA found
- No physical evidence
- No corpse ever found
- Loch ecology couldn’t support large predator population
10. Conclusion
- H₀ cannot be rejected
- Most likely explanation: Misidentifications, hoaxes, wishful thinking
- Wave patterns, logs, otters, birds can appear monster-like
- Psychological factors (pareidolia, confirmation bias)
11. Scientific Consensus
- No credible evidence for Loch Ness Monster
- Sightings explainable by known phenomena
- Case demonstrates importance of methodology
Lessons
- Anecdotes ≠ evidence
- Need systematic investigation
- Burden of proof on extraordinary claims
- Methodology prevents self-deception
You want to investigate whether young or old people speak more fluently.
How would you design this study?
Study design
1. Define Terms
“Fluency” could mean
- Speed (words per minute)
- Hesitations (um, uh, pauses)
- Errors (false starts, repairs)
- Coherence (staying on topic)
Choose operational definition (e.g., words per minute with minimal hesitations)
2. Define “Young” and “Old”
- Young: 20-30 years
- Old: 65-75 years
- (Specific age ranges needed)
3. Form Hypotheses
H₁ Younger speakers are more fluent
H₀ No age difference in fluency
Or reverse H₁ Older speakers are more fluent (more practice!)
4. Sample
- Need: 30+ participants per group (power analysis)
- Match on: Education, health status, language background
- Screen for: Speech disorders, cognitive impairment
- Recruit: Community sampling, advertisements
5. Task
- Picture description
- Narrative retelling
- Semi-structured interview
- (Standardized task needed)
6. Measure
- Record speech
- Transcribe
- Count:
- Words per minute
- Filled pauses (um, uh)
- Silent pauses >1 second
- Self-corrections
- False starts
7. Control
- Same task for all participants
- Same instructions
- Same recording conditions
- Blind coding (coder doesn’t know age)
8. Analysis
- Compare groups on fluency measures
- T-tests or ANOVA
- Effect sizes
- Control for confounds (education, etc.)
9. Potential Confounds
- Health differences
- Hearing ability
- Topic familiarity
- Testing anxiety
- Cohort effects
10. Expected Results
Could find
- Younger faster but more disfluent (speed-accuracy tradeoff)
- Older slower but more fluent (practice effect)
- No difference (compensating factors)
- Task-dependent effects
Important Need operational definitions and controls!
Claim “Vitamin X cures cancer!”
Scientific questions
1. Is it falsifiable? (Yes—could test on cancer patients)
2. What’s the evidence?
- Anecdotes? (Not sufficient)
- Observational studies? (Better, but confounds)
- Randomized controlled trials? (Gold standard)
3. Sample size adequate?
4. Proper controls?
5. Conflicts of interest?
6. Published in peer-reviewed journal?
7. Replicated by independent researchers?
Headline “Study shows chocolate improves memory!”
Critical questions
1. Correlation or causation?
2. Sample size?
3. Control group?
4. Effect size (meaningful or trivial)?
5. Confounds controlled?
6. Who funded study? (Chocolate industry?)
7. Published where?
8. Consistent with other research?
Scenario Should I buy this “quantum healing bracelet”?
Scientific analysis
1. Claim Bracelet balances body’s quantum energy
2. Falsifiable? Vague, not testable
3. Mechanism No plausible biological mechanism
4. Evidence Only testimonials (anecdotes)
5. Alternative explanations Placebo effect
6. Red flags
- Pseudoscientific language
- Appeals to quantum physics (misused)
- Only alternative medicine sites promote
7. Conclusion Extremely unlikely to work via claimed mechanism
When making decisions, watch for
1. Observe → 2. Question → 3. Review literature →
4. Hypothesize → 5. Design → 6. Collect data →
7. Analyze → 8. Conclude → 9. Refine → [Repeat] Key principles
- Falsifiable hypotheses
- Controlled experiments
- Systematic observation
- Statistical analysis
- Peer review
- Replication
Questions to ask
Schweinberger, Martin. 2026. Introduction to Quantitative Reasoning: Why We Need Science. Brisbane: The Language Technology and Data Analysis Laboratory (LADAL). url: https://ladal.edu.au/tutorials/introquant.html (Version 2026.02.10).
@manual{schweinberger2026introquant,
author = {Schweinberger, Martin},
title = {Introduction to Quantitative Reasoning: Why We Need Science},
note = {https://ladal.edu.au/tutorials/introquant.html},
year = {2026},
organization = {The Language Technology and Data Analysis Laboratory (LADAL)},
address = {Brisbane},
edition = {2026.02.10}
} sessionInfo() R version 4.4.2 (2024-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 26200)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.utf8
[2] LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8
time zone: Australia/Brisbane
tzcode source: internal
attached base packages:
[1] stats graphics grDevices datasets utils methods base
other attached packages:
[1] tufte_0.13 cowplot_1.2.0 magick_2.8.5 lubridate_1.9.4
[5] forcats_1.0.0 stringr_1.5.1 dplyr_1.2.0 purrr_1.0.4
[9] readr_2.1.5 tidyr_1.3.2 tibble_3.2.1 ggplot2_4.0.2
[13] tidyverse_2.0.0 flextable_0.9.7 knitr_1.51
loaded via a namespace (and not attached):
[1] generics_0.1.3 fontLiberation_0.1.0 renv_1.1.1
[4] xml2_1.3.6 stringi_1.8.4 hms_1.1.3
[7] digest_0.6.39 magrittr_2.0.3 timechange_0.3.0
[10] evaluate_1.0.3 grid_4.4.2 RColorBrewer_1.1-3
[13] fastmap_1.2.0 jsonlite_1.9.0 zip_2.3.2
[16] scales_1.4.0 fontBitstreamVera_0.1.1 codetools_0.2-20
[19] textshaping_1.0.0 cli_3.6.4 rlang_1.1.7
[22] fontquiver_0.2.1 withr_3.0.2 yaml_2.3.10
[25] gdtools_0.4.1 tools_4.4.2 officer_0.6.7
[28] tzdb_0.4.0 uuid_1.2-1 vctrs_0.7.1
[31] R6_2.6.1 lifecycle_1.0.5 htmlwidgets_1.6.4
[34] ragg_1.3.3 pkgconfig_2.0.3 pillar_1.10.1
[37] gtable_0.3.6 data.table_1.17.0 glue_1.8.0
[40] Rcpp_1.0.14 systemfonts_1.2.1 xfun_0.56
[43] tidyselect_1.2.1 rstudioapi_0.17.1 farver_2.1.2
[46] htmltools_0.5.9 labeling_0.4.3 rmarkdown_2.30
[49] compiler_4.4.2 S7_0.2.1 askpass_1.2.1
[52] openssl_2.3.2